1. Field of the Invention
The present invention relates to systems and methods for scoring test answers from optical images of answer pages, and, more particularly, to systems and methods for improving a work flow for such scoring applications.
2. Description of Related Art
The automation of test scoring is a complex problem that has brought to bear significant economic pressure to optimize efficiency and accuracy and to minimize human involvement.
The scanning and data collection from test answer sheets by visual imaging means is also known in the art, for example, in commonly owned U.S. Pat. Nos. 6,173,154, 6,311,040, and 6,366,760, the disclosures of which are incorporated hereinto by reference. These patents teach a combination of OMR and visual imaging for capturing a full visual image of each answer page containing an answer to an open-ended question.
When large numbers of tests are to be graded at a scoring center, typically groups of physical test booklets are retained together based upon a particular criterion, such as individual grade levels from a particular school or school district, and a predetermined number are placed on a cart. Each test booklet is separated into individual answer sheets, and the cart is moved to a scanning area. The individual answer sheets are then sent through a scanner, which creates a scanner output record for each test booklet. The record contains such data as identifier information and test answer data gleaned from the answer sheets.
The complete system includes integrated hardware elements and software applications for capturing optical mark and full visual images of an answer page, for storing the images, for retrieving the images, for distributing the visual images to a reader for scoring, for assisting the reader in scoring, and for monitoring the reader's performance.
The scanning system comprises means for sequentially advancing each page of a plurality of answer pages along a predetermined path. Positioned along the path are mark imaging means (OMR, optical mark recognition; OCR, optical character recognition) for capturing a location of an optical mark on each answer page and visual imaging means for capturing a full visual image of each answer page. A forms database in a server is provided that contains data on the physical location and type (e.g., multiple-choice or open-ended) of each answer on each page. Software means resident in the server operate with the forms database to determine whether the captured image contains an answer to an open-ended question. If such an open-ended answer is supposed to be found on the page being imaged, the full visual image of the page is stored.
At present, if a scorer receives an optical image of an answer page that is of insufficient quality to be read, the physical answer page must be requested from storage. Then the physical answer page is distributed to the scorer for scoring. If a particular answer booklet contains more than one page having insufficient image quality, each request for a page is handled separately, meaning that the same booklet is the subject of multiple search efforts. Typically, the cart of physical test booklets will by this time have been removed from the scanning area and archived. Clearly the handling of such multiple requests disturbs scoring work flow. Further, frequently the scoring customer will have requested image data along with the assigned scores. If any of these answer page images is of poor quality, the customer cannot adequately review the scored answer.
In order to present the problem addressed by the present invention in detail, FIGS. 1A–1D are presented detailing the method steps currently known in the art to be performed to provide a readable answer page to a scorer. In the prior art method 100, if a scorer receives a poor-quality image of an answer page to be scored at a scoring site (block 101), the scorer transmits a request up the management chain for a readable copy of the answer page. Such a request is routed to a scoring support area (block 102), and scoring on the batch from which the answer page came is halted (block 103). A report containing the request is printed (block 104), and the physical report is routed to an archiving center (block 105), where the physical booklet containing the answer page is retrieved (block 106). A photocopy of the answer page is made and sent to a scoring storeroom (block 107), where the copy is matched with the request report (block 108).
At the scoring storeroom, the hard copy is checked for acceptable quality (block 109). If the hard copy still has problems, it is sent to an “alert” center, the responsibility of which is to research and repair answer booklets (block 110). The repaired booklets are then compiled (block 111), and the booklets are returned to the storeroom (block 112).
If the hard copy is acceptable, it is routed from the storeroom to the scoring center and delivered to the scorer (block 113). The scorer assigns a score using the hard copy (block 114), and the hard copy is returned to the storeroom (block 115), where it is retained until scoring on that batch is complete (block 116).
If the answer page belongs to a booklet that has been repaired (block 117), the repaired booklet is transmitted to the alert center (block 118). If electronic images have been requested by the customer (block 119), the hard copy of the answer page is transmitted to a scanning center (block 122), where all documents are re-scanned (block 123). The original images are manually deleted from the database (block 124), and an image index is manually re-created (block 125), and the new images are imported into the database (block 126). The hard copy is then returned to the storeroom (block 120) and from there to the archive center (block 121), and the process is complete.
If the answer page belongs to a booklet that has not been repaired (block 117), or if electronic images have not been requested (block 119), then, as above, the hard copy is returned to the storeroom (block 120) and from there to the archive center (block 121), and the process is complete.
It can be seen that six different locations are involved in this process: the scoring center, the scanning support center, the archive center, the storeroom, the scanning center, and the alerts center. Further, a hard copy of the problem answer page has to be physically routed through all these centers. This is clearly a labor- and time-intensive process that can cause significant delays in completing a scoring project.